Assignment1_Data621

Load Libraries

Data Exploration

Loading Datasets

eval_data <- read.csv("https://raw.githubusercontent.com/uzmabb182/Data_621/refs/heads/main/Assignment1/moneyball-evaluation-data%20(1).csv")

train_data <- read.csv("https://raw.githubusercontent.com/uzmabb182/Data_621/refs/heads/main/Assignment1/moneyball-training-data%20(1).csv")

head(eval_data)

head(train_data)

Understanding the TARGET_WINS Field in the Moneyball Dataset:

The TARGET_WINS field represents the number of games a team won in a given baseball season.

This is the dependent variable (target variable) in the multiple linear regression model.

We are trying to predict TARGET_WINS based on other team performance metrics.

Viewing Column Names

names(eval_data)

##  [1] "INDEX"            "TEAM_BATTING_H"   "TEAM_BATTING_2B"  "TEAM_BATTING_3B" 
##  [5] "TEAM_BATTING_HR"  "TEAM_BATTING_BB"  "TEAM_BATTING_SO"  "TEAM_BASERUN_SB" 
##  [9] "TEAM_BASERUN_CS"  "TEAM_BATTING_HBP" "TEAM_PITCHING_H"  "TEAM_PITCHING_HR"
## [13] "TEAM_PITCHING_BB" "TEAM_PITCHING_SO" "TEAM_FIELDING_E"  "TEAM_FIELDING_DP"

names(train_data)

##  [1] "INDEX"            "TARGET_WINS"      "TEAM_BATTING_H"   "TEAM_BATTING_2B" 
##  [5] "TEAM_BATTING_3B"  "TEAM_BATTING_HR"  "TEAM_BATTING_BB"  "TEAM_BATTING_SO" 
##  [9] "TEAM_BASERUN_SB"  "TEAM_BASERUN_CS"  "TEAM_BATTING_HBP" "TEAM_PITCHING_H" 
## [13] "TEAM_PITCHING_HR" "TEAM_PITCHING_BB" "TEAM_PITCHING_SO" "TEAM_FIELDING_E" 
## [17] "TEAM_FIELDING_DP"

Viewing Field Values:

We can see there are fields with null values

glimpse(eval_data)

## Rows: 259
## Columns: 16
## $ INDEX            <int> 9, 10, 14, 47, 60, 63, 74, 83, 98, 120, 123, 135, 138…
## $ TEAM_BATTING_H   <int> 1209, 1221, 1395, 1539, 1445, 1431, 1430, 1385, 1259,…
## $ TEAM_BATTING_2B  <int> 170, 151, 183, 309, 203, 236, 219, 158, 177, 212, 243…
## $ TEAM_BATTING_3B  <int> 33, 29, 29, 29, 68, 53, 55, 42, 78, 42, 40, 55, 57, 2…
## $ TEAM_BATTING_HR  <int> 83, 88, 93, 159, 5, 10, 37, 33, 23, 58, 50, 164, 186,…
## $ TEAM_BATTING_BB  <int> 447, 516, 509, 486, 95, 215, 568, 356, 466, 452, 495,…
## $ TEAM_BATTING_SO  <int> 1080, 929, 816, 914, 416, 377, 527, 609, 689, 584, 64…
## $ TEAM_BASERUN_SB  <int> 62, 54, 59, 148, NA, NA, 365, 185, 150, 52, 64, 48, 3…
## $ TEAM_BASERUN_CS  <int> 50, 39, 47, 57, NA, NA, NA, NA, NA, NA, NA, 28, 21, 8…
## $ TEAM_BATTING_HBP <int> NA, NA, NA, 42, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…
## $ TEAM_PITCHING_H  <int> 1209, 1221, 1395, 1539, 3902, 2793, 1544, 1626, 1342,…
## $ TEAM_PITCHING_HR <int> 83, 88, 93, 159, 14, 20, 40, 39, 25, 62, 53, 173, 196…
## $ TEAM_PITCHING_BB <int> 447, 516, 509, 486, 257, 420, 613, 418, 497, 482, 521…
## $ TEAM_PITCHING_SO <int> 1080, 929, 816, 914, 1123, 736, 569, 715, 734, 622, 6…
## $ TEAM_FIELDING_E  <int> 140, 135, 156, 124, 616, 572, 490, 328, 226, 184, 200…
## $ TEAM_FIELDING_DP <int> 156, 164, 153, 154, 130, 105, NA, 104, 132, 145, 183,…

glimpse(train_data)

## Rows: 2,276
## Columns: 17
## $ INDEX            <int> 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 17, 18, 1…
## $ TARGET_WINS      <int> 39, 70, 86, 70, 82, 75, 80, 85, 86, 76, 78, 68, 72, 7…
## $ TEAM_BATTING_H   <int> 1445, 1339, 1377, 1387, 1297, 1279, 1244, 1273, 1391,…
## $ TEAM_BATTING_2B  <int> 194, 219, 232, 209, 186, 200, 179, 171, 197, 213, 179…
## $ TEAM_BATTING_3B  <int> 39, 22, 35, 38, 27, 36, 54, 37, 40, 18, 27, 31, 41, 2…
## $ TEAM_BATTING_HR  <int> 13, 190, 137, 96, 102, 92, 122, 115, 114, 96, 82, 95,…
## $ TEAM_BATTING_BB  <int> 143, 685, 602, 451, 472, 443, 525, 456, 447, 441, 374…
## $ TEAM_BATTING_SO  <int> 842, 1075, 917, 922, 920, 973, 1062, 1027, 922, 827, …
## $ TEAM_BASERUN_SB  <int> NA, 37, 46, 43, 49, 107, 80, 40, 69, 72, 60, 119, 221…
## $ TEAM_BASERUN_CS  <int> NA, 28, 27, 30, 39, 59, 54, 36, 27, 34, 39, 79, 109, …
## $ TEAM_BATTING_HBP <int> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…
## $ TEAM_PITCHING_H  <int> 9364, 1347, 1377, 1396, 1297, 1279, 1244, 1281, 1391,…
## $ TEAM_PITCHING_HR <int> 84, 191, 137, 97, 102, 92, 122, 116, 114, 96, 86, 95,…
## $ TEAM_PITCHING_BB <int> 927, 689, 602, 454, 472, 443, 525, 459, 447, 441, 391…
## $ TEAM_PITCHING_SO <int> 5456, 1082, 917, 928, 920, 973, 1062, 1033, 922, 827,…
## $ TEAM_FIELDING_E  <int> 1011, 193, 175, 164, 138, 123, 136, 112, 127, 131, 11…
## $ TEAM_FIELDING_DP <int> NA, 155, 153, 156, 168, 149, 186, 136, 169, 159, 141,…

Creating Dataframe

eval_df <- data.frame(eval_data)
train_df <- data.frame(train_data)
train_df

str(eval_df)

## 'data.frame':    259 obs. of  16 variables:
##  $ INDEX           : int  9 10 14 47 60 63 74 83 98 120 ...
##  $ TEAM_BATTING_H  : int  1209 1221 1395 1539 1445 1431 1430 1385 1259 1397 ...
##  $ TEAM_BATTING_2B : int  170 151 183 309 203 236 219 158 177 212 ...
##  $ TEAM_BATTING_3B : int  33 29 29 29 68 53 55 42 78 42 ...
##  $ TEAM_BATTING_HR : int  83 88 93 159 5 10 37 33 23 58 ...
##  $ TEAM_BATTING_BB : int  447 516 509 486 95 215 568 356 466 452 ...
##  $ TEAM_BATTING_SO : int  1080 929 816 914 416 377 527 609 689 584 ...
##  $ TEAM_BASERUN_SB : int  62 54 59 148 NA NA 365 185 150 52 ...
##  $ TEAM_BASERUN_CS : int  50 39 47 57 NA NA NA NA NA NA ...
##  $ TEAM_BATTING_HBP: int  NA NA NA 42 NA NA NA NA NA NA ...
##  $ TEAM_PITCHING_H : int  1209 1221 1395 1539 3902 2793 1544 1626 1342 1489 ...
##  $ TEAM_PITCHING_HR: int  83 88 93 159 14 20 40 39 25 62 ...
##  $ TEAM_PITCHING_BB: int  447 516 509 486 257 420 613 418 497 482 ...
##  $ TEAM_PITCHING_SO: int  1080 929 816 914 1123 736 569 715 734 622 ...
##  $ TEAM_FIELDING_E : int  140 135 156 124 616 572 490 328 226 184 ...
##  $ TEAM_FIELDING_DP: int  156 164 153 154 130 105 NA 104 132 145 ...

str(train_df)

## 'data.frame':    2276 obs. of  17 variables:
##  $ INDEX           : int  1 2 3 4 5 6 7 8 11 12 ...
##  $ TARGET_WINS     : int  39 70 86 70 82 75 80 85 86 76 ...
##  $ TEAM_BATTING_H  : int  1445 1339 1377 1387 1297 1279 1244 1273 1391 1271 ...
##  $ TEAM_BATTING_2B : int  194 219 232 209 186 200 179 171 197 213 ...
##  $ TEAM_BATTING_3B : int  39 22 35 38 27 36 54 37 40 18 ...
##  $ TEAM_BATTING_HR : int  13 190 137 96 102 92 122 115 114 96 ...
##  $ TEAM_BATTING_BB : int  143 685 602 451 472 443 525 456 447 441 ...
##  $ TEAM_BATTING_SO : int  842 1075 917 922 920 973 1062 1027 922 827 ...
##  $ TEAM_BASERUN_SB : int  NA 37 46 43 49 107 80 40 69 72 ...
##  $ TEAM_BASERUN_CS : int  NA 28 27 30 39 59 54 36 27 34 ...
##  $ TEAM_BATTING_HBP: int  NA NA NA NA NA NA NA NA NA NA ...
##  $ TEAM_PITCHING_H : int  9364 1347 1377 1396 1297 1279 1244 1281 1391 1271 ...
##  $ TEAM_PITCHING_HR: int  84 191 137 97 102 92 122 116 114 96 ...
##  $ TEAM_PITCHING_BB: int  927 689 602 454 472 443 525 459 447 441 ...
##  $ TEAM_PITCHING_SO: int  5456 1082 917 928 920 973 1062 1033 922 827 ...
##  $ TEAM_FIELDING_E : int  1011 193 175 164 138 123 136 112 127 131 ...
##  $ TEAM_FIELDING_DP: int  NA 155 153 156 168 149 186 136 169 159 ...

train_df %>%
  gather(variable, value, -TARGET_WINS) %>%
  ggplot(., aes(value, TARGET_WINS)) + 
  geom_point(fill = "#628B3A", color="#628B3A")  + 
  geom_smooth(method = "lm", se = FALSE, color = "black") + 
  facet_wrap(~variable, scales ="free", ncol = 4) +
  labs(x = element_blank(), y = "Wins")

## `geom_smooth()` using formula = 'y ~ x'

## Warning: Removed 3478 rows containing non-finite outside the scale range
## (`stat_smooth()`).

## Warning: Removed 3478 rows containing missing values or values outside the scale range
## (`geom_point()`).

### Sorted Correlations

correlation_with_target <- cor(train_df, use = "complete.obs")["TARGET_WINS", ] %>%
  sort(decreasing = TRUE)  # Sort from highest to lowest correlation
print(correlation_with_target)

##      TARGET_WINS  TEAM_PITCHING_H   TEAM_BATTING_H  TEAM_BATTING_BB 
##       1.00000000       0.47123431       0.46994665       0.46868793 
## TEAM_PITCHING_BB TEAM_PITCHING_HR  TEAM_BATTING_HR  TEAM_BATTING_2B 
##       0.46839882       0.42246683       0.42241683       0.31298400 
## TEAM_BATTING_HBP  TEAM_BASERUN_SB            INDEX  TEAM_BATTING_3B 
##       0.07350424       0.01483639      -0.04895047      -0.12434586 
##  TEAM_BASERUN_CS TEAM_FIELDING_DP  TEAM_BATTING_SO TEAM_PITCHING_SO 
##      -0.17875598      -0.19586601      -0.22889273      -0.22936481 
##  TEAM_FIELDING_E 
##      -0.38668800

library(ggplot2)

correlation_data <- data.frame(Variable = names(correlation_with_target), Correlation = correlation_with_target)

ggplot(correlation_data, aes(x = reorder(Variable, Correlation), y = Correlation, fill = Correlation > 0)) +
  geom_bar(stat = "identity") +
  coord_flip() +  # Flip for better readability
  labs(title = "Correlation with Target Wins", x = "Variables", y = "Correlation") +
  scale_fill_manual(values = c("red", "blue")) +
  theme_minimal()

### Checking Correlations and Finding Action:

Strong positive correlation (> 0.5) -> keep the variable in regression.

Strong negative correlation (< -0.5) -> Keep (inverse relationship).

Weak correlation (~0) -> Consider removing.

Two highly correlated variables (> 0.85) -> Drop one to avoid multicollinearity.

Summary of Datasets

skim(eval_df)

Data summary
Name	eval_df
Number of rows	259
Number of columns	16
_______________________
Column type frequency:
numeric	16
________________________
Group variables	None

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
INDEX	0	1.00	1263.93	693.29	9	708.0	1249.0	1832.50	2525	▆▆▆▇▅
TEAM_BATTING_H	0	1.00	1469.39	150.66	819	1387.0	1455.0	1548.00	2170	▁▂▇▁▁
TEAM_BATTING_2B	0	1.00	241.32	49.52	44	210.0	239.0	278.50	376	▁▂▇▇▂
TEAM_BATTING_3B	0	1.00	55.91	27.14	14	35.0	52.0	72.00	155	▇▇▃▁▁
TEAM_BATTING_HR	0	1.00	95.63	56.33	0	44.5	101.0	135.50	242	▆▅▇▃▁
TEAM_BATTING_BB	0	1.00	498.96	120.59	15	436.5	509.0	565.50	792	▁▁▅▇▁
TEAM_BATTING_SO	18	0.93	709.34	243.11	0	545.0	686.0	912.00	1268	▁▃▇▇▂
TEAM_BASERUN_SB	13	0.95	123.70	93.39	0	59.0	92.0	151.75	580	▇▃▁▁▁
TEAM_BASERUN_CS	87	0.66	52.32	23.10	0	38.0	49.5	63.00	154	▂▇▃▁▁
TEAM_BATTING_HBP	240	0.07	62.37	12.71	42	53.5	62.0	67.50	96	▃▇▅▁▁
TEAM_PITCHING_H	0	1.00	1813.46	1662.91	1155	1426.5	1515.0	1681.00	22768	▇▁▁▁▁
TEAM_PITCHING_HR	0	1.00	102.15	57.65	0	52.0	104.0	142.50	336	▇▇▆▁▁
TEAM_PITCHING_BB	0	1.00	552.42	172.95	136	471.0	526.0	606.50	2008	▆▇▁▁▁
TEAM_PITCHING_SO	18	0.93	799.67	634.31	0	613.0	745.0	938.00	9963	▇▁▁▁▁
TEAM_FIELDING_E	0	1.00	249.75	230.90	73	131.0	163.0	252.00	1568	▇▁▁▁▁
TEAM_FIELDING_DP	31	0.88	146.06	25.88	69	131.0	148.0	164.00	204	▁▂▇▇▂

skim(train_df)

Data summary
Name	train_df
Number of rows	2276
Number of columns	17
_______________________
Column type frequency:
numeric	17
________________________
Group variables	None

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
INDEX	0	1.00	1268.46	736.35	1	630.75	1270.5	1915.50	2535	▇▇▇▇▇
TARGET_WINS	0	1.00	80.79	15.75	0	71.00	82.0	92.00	146	▁▁▇▅▁
TEAM_BATTING_H	0	1.00	1469.27	144.59	891	1383.00	1454.0	1537.25	2554	▁▇▂▁▁
TEAM_BATTING_2B	0	1.00	241.25	46.80	69	208.00	238.0	273.00	458	▁▆▇▂▁
TEAM_BATTING_3B	0	1.00	55.25	27.94	0	34.00	47.0	72.00	223	▇▇▂▁▁
TEAM_BATTING_HR	0	1.00	99.61	60.55	0	42.00	102.0	147.00	264	▇▆▇▅▁
TEAM_BATTING_BB	0	1.00	501.56	122.67	0	451.00	512.0	580.00	878	▁▁▇▇▁
TEAM_BATTING_SO	102	0.96	735.61	248.53	0	548.00	750.0	930.00	1399	▁▆▇▇▁
TEAM_BASERUN_SB	131	0.94	124.76	87.79	0	66.00	101.0	156.00	697	▇▃▁▁▁
TEAM_BASERUN_CS	772	0.66	52.80	22.96	0	38.00	49.0	62.00	201	▃▇▁▁▁
TEAM_BATTING_HBP	2085	0.08	59.36	12.97	29	50.50	58.0	67.00	95	▂▇▇▅▁
TEAM_PITCHING_H	0	1.00	1779.21	1406.84	1137	1419.00	1518.0	1682.50	30132	▇▁▁▁▁
TEAM_PITCHING_HR	0	1.00	105.70	61.30	0	50.00	107.0	150.00	343	▇▇▆▁▁
TEAM_PITCHING_BB	0	1.00	553.01	166.36	0	476.00	536.5	611.00	3645	▇▁▁▁▁
TEAM_PITCHING_SO	102	0.96	817.73	553.09	0	615.00	813.5	968.00	19278	▇▁▁▁▁
TEAM_FIELDING_E	0	1.00	246.48	227.77	65	127.00	159.0	249.25	1898	▇▁▁▁▁
TEAM_FIELDING_DP	286	0.87	146.39	26.23	52	131.00	149.0	164.00	228	▁▂▇▆▁

summary(train_df)

##      INDEX         TARGET_WINS     TEAM_BATTING_H TEAM_BATTING_2B
##  Min.   :   1.0   Min.   :  0.00   Min.   : 891   Min.   : 69.0  
##  1st Qu.: 630.8   1st Qu.: 71.00   1st Qu.:1383   1st Qu.:208.0  
##  Median :1270.5   Median : 82.00   Median :1454   Median :238.0  
##  Mean   :1268.5   Mean   : 80.79   Mean   :1469   Mean   :241.2  
##  3rd Qu.:1915.5   3rd Qu.: 92.00   3rd Qu.:1537   3rd Qu.:273.0  
##  Max.   :2535.0   Max.   :146.00   Max.   :2554   Max.   :458.0  
##                                                                  
##  TEAM_BATTING_3B  TEAM_BATTING_HR  TEAM_BATTING_BB TEAM_BATTING_SO 
##  Min.   :  0.00   Min.   :  0.00   Min.   :  0.0   Min.   :   0.0  
##  1st Qu.: 34.00   1st Qu.: 42.00   1st Qu.:451.0   1st Qu.: 548.0  
##  Median : 47.00   Median :102.00   Median :512.0   Median : 750.0  
##  Mean   : 55.25   Mean   : 99.61   Mean   :501.6   Mean   : 735.6  
##  3rd Qu.: 72.00   3rd Qu.:147.00   3rd Qu.:580.0   3rd Qu.: 930.0  
##  Max.   :223.00   Max.   :264.00   Max.   :878.0   Max.   :1399.0  
##                                                    NA's   :102     
##  TEAM_BASERUN_SB TEAM_BASERUN_CS TEAM_BATTING_HBP TEAM_PITCHING_H
##  Min.   :  0.0   Min.   :  0.0   Min.   :29.00    Min.   : 1137  
##  1st Qu.: 66.0   1st Qu.: 38.0   1st Qu.:50.50    1st Qu.: 1419  
##  Median :101.0   Median : 49.0   Median :58.00    Median : 1518  
##  Mean   :124.8   Mean   : 52.8   Mean   :59.36    Mean   : 1779  
##  3rd Qu.:156.0   3rd Qu.: 62.0   3rd Qu.:67.00    3rd Qu.: 1682  
##  Max.   :697.0   Max.   :201.0   Max.   :95.00    Max.   :30132  
##  NA's   :131     NA's   :772     NA's   :2085                    
##  TEAM_PITCHING_HR TEAM_PITCHING_BB TEAM_PITCHING_SO  TEAM_FIELDING_E 
##  Min.   :  0.0    Min.   :   0.0   Min.   :    0.0   Min.   :  65.0  
##  1st Qu.: 50.0    1st Qu.: 476.0   1st Qu.:  615.0   1st Qu.: 127.0  
##  Median :107.0    Median : 536.5   Median :  813.5   Median : 159.0  
##  Mean   :105.7    Mean   : 553.0   Mean   :  817.7   Mean   : 246.5  
##  3rd Qu.:150.0    3rd Qu.: 611.0   3rd Qu.:  968.0   3rd Qu.: 249.2  
##  Max.   :343.0    Max.   :3645.0   Max.   :19278.0   Max.   :1898.0  
##                                    NA's   :102                       
##  TEAM_FIELDING_DP
##  Min.   : 52.0   
##  1st Qu.:131.0   
##  Median :149.0   
##  Mean   :146.4   
##  3rd Qu.:164.0   
##  Max.   :228.0   
##  NA's   :286

summary(eval_df)

##      INDEX      TEAM_BATTING_H TEAM_BATTING_2B TEAM_BATTING_3B 
##  Min.   :   9   Min.   : 819   Min.   : 44.0   Min.   : 14.00  
##  1st Qu.: 708   1st Qu.:1387   1st Qu.:210.0   1st Qu.: 35.00  
##  Median :1249   Median :1455   Median :239.0   Median : 52.00  
##  Mean   :1264   Mean   :1469   Mean   :241.3   Mean   : 55.91  
##  3rd Qu.:1832   3rd Qu.:1548   3rd Qu.:278.5   3rd Qu.: 72.00  
##  Max.   :2525   Max.   :2170   Max.   :376.0   Max.   :155.00  
##                                                                
##  TEAM_BATTING_HR  TEAM_BATTING_BB TEAM_BATTING_SO  TEAM_BASERUN_SB
##  Min.   :  0.00   Min.   : 15.0   Min.   :   0.0   Min.   :  0.0  
##  1st Qu.: 44.50   1st Qu.:436.5   1st Qu.: 545.0   1st Qu.: 59.0  
##  Median :101.00   Median :509.0   Median : 686.0   Median : 92.0  
##  Mean   : 95.63   Mean   :499.0   Mean   : 709.3   Mean   :123.7  
##  3rd Qu.:135.50   3rd Qu.:565.5   3rd Qu.: 912.0   3rd Qu.:151.8  
##  Max.   :242.00   Max.   :792.0   Max.   :1268.0   Max.   :580.0  
##                                   NA's   :18       NA's   :13     
##  TEAM_BASERUN_CS  TEAM_BATTING_HBP TEAM_PITCHING_H TEAM_PITCHING_HR
##  Min.   :  0.00   Min.   :42.00    Min.   : 1155   Min.   :  0.0   
##  1st Qu.: 38.00   1st Qu.:53.50    1st Qu.: 1426   1st Qu.: 52.0   
##  Median : 49.50   Median :62.00    Median : 1515   Median :104.0   
##  Mean   : 52.32   Mean   :62.37    Mean   : 1813   Mean   :102.1   
##  3rd Qu.: 63.00   3rd Qu.:67.50    3rd Qu.: 1681   3rd Qu.:142.5   
##  Max.   :154.00   Max.   :96.00    Max.   :22768   Max.   :336.0   
##  NA's   :87       NA's   :240                                      
##  TEAM_PITCHING_BB TEAM_PITCHING_SO TEAM_FIELDING_E  TEAM_FIELDING_DP
##  Min.   : 136.0   Min.   :   0.0   Min.   :  73.0   Min.   : 69.0   
##  1st Qu.: 471.0   1st Qu.: 613.0   1st Qu.: 131.0   1st Qu.:131.0   
##  Median : 526.0   Median : 745.0   Median : 163.0   Median :148.0   
##  Mean   : 552.4   Mean   : 799.7   Mean   : 249.7   Mean   :146.1   
##  3rd Qu.: 606.5   3rd Qu.: 938.0   3rd Qu.: 252.0   3rd Qu.:164.0   
##  Max.   :2008.0   Max.   :9963.0   Max.   :1568.0   Max.   :204.0   
##                   NA's   :18                        NA's   :31

Calculate mean, median, and standard deviation for key numerical columns

train_df %>%
  summarise(across(where(is.numeric), list(mean = mean, median = median, sd = sd), na.rm = TRUE))

## Warning: There was 1 warning in `summarise()`.
## ℹ In argument: `across(...)`.
## Caused by warning:
## ! The `...` argument of `across()` is deprecated as of dplyr 1.1.0.
## Supply arguments directly to `.fns` through an anonymous function instead.
## 
##   # Previously
##   across(a:b, mean, na.rm = TRUE)
## 
##   # Now
##   across(a:b, \(x) mean(x, na.rm = TRUE))

The summarise(across()) function above is calculating the mean, median, ans standard deviation for all the numerical variables.

Analysis of the training Dataset Before Fitting a Multiple Linear Regression Model:

Before fitting a multiple linear regression (MLR) model, we analyze the dataset for potential issues such as missing values, extreme outliers, multicollinearity, and variable distributions.

Here’s what we can infer from the summary statistics:

1. Understanding the Target Variable (TARGET_WINS)

Range: 0 to 146 wins

Mean: ~80.79 wins

Median: 82 wins

Distribution: The min value of 0 and max of 146 suggest some potential outliers or erroneous data points, since most teams win between 50-110 games in a season.

2. Missing Values

Some variables have a significant number of missing values:

TEAM_BATTING_SO (102 missing values)

TEAM_BASERUN_SB (131 missing values)

TEAM_BASERUN_CS (772 missing values) <- potentially unreliable

TEAM_BATTING_HBP (2085 missing values) <- very unreliable

TEAM_PITCHING_SO (102 missing values)

TEAM_FIELDING_DP (286 missing values)

Actionable Steps:

Impute missing values (using mean/median or regression techniques).
Consider removing TEAM_BATTING_HBP and TEAM_BASERUN_CS if they are highly incomplete and do not contribute much.

3. Potential Outliers & Data Issues

Several variables have extreme max values that seem unrealistic:

TEAM_PITCHING_H (Max = 30,132) <- Likely an error since typical values range from 1,200 - 1,700.

TEAM_PITCHING_SO (Max = 19,278) <- Suspiciously high (typical range: 500 - 1,500).

TEAM_PITCHING_BB (Max = 3,645) <- Very high (typical range: 300 - 700).

TEAM_FIELDING_E (Max = 1,898) <- Likely an error since the normal range is ~ 70-200.

Actionable Steps:

Check for data entry errors.
Remove extreme outliers if they distort model performance.

4. Feature Selection Considerations:

Batting Variables: TEAM_BATTING_H, TEAM_BATTING_2B, TEAM_BATTING_3B, TEAM_BATTING_HR, TEAM_BATTING_BB are likely strong predictors of team wins.

Pitching Variables: TEAM_PITCHING_H, TEAM_PITCHING_HR, TEAM_PITCHING_BB, TEAM_PITCHING_SO will impact defensive strength.

Fielding Variables: TEAM_FIELDING_E (errors) and TEAM_FIELDING_DP (double plays) may have a weaker impact compared to batting and pitching.

Multicollinearity Check Needed:

TEAM_PITCHING_H, TEAM_PITCHING_HR, and TEAM_PITCHING_BB may be highly correlated, which can cause multicollinearity in the regression model.

TEAM_BATTING_H, TEAM_BATTING_2B, and TEAM_BATTING_3B may also be strongly correlated since total hits include doubles and triples.

5. Data Cleaning Recommendations Before Fitting the Model

Handle Missing Values

Consider dropping or imputing variables with too many missing values (e.g., TEAM_BATTING_HBP).

Impute TEAM_BASERUN_SB, TEAM_BASERUN_CS, and TEAM_FIELDING_DP appropriately.

Remove or Adjust Extreme Outliers

Remove highly unrealistic values in pitching, fielding, and errors.

Check for Multicollinearity

Use Variance Inflation Factor (VIF) to detect multicollinearity and drop redundant features.

Feature Engineering

Consider derived metrics like batting average (H/AB), on-base percentage (OBP), or earned run average (ERA) instead of raw counts.

Conclusion:

The dataset contains inconsistencies, missing values, and extreme outliers that need to be addressed before fitting an MLR model.

Once cleaned, feature selection and multicollinearity checks will be essential to ensure a robust and interpretable model for predicting team wins.

Visualizing Training Dataset:

boxplot(train_df$TARGET_WINS, main="Distribution of Team Wins", ylab="Wins", col="lightblue")

Analysis of the Box Plot for TARGET_WINS (Team Wins Distribution)

This box plot provides valuable insights into the distribution of team wins in the training dataset.

Here’s what we can infer:

1. Median and Spread of Wins

The thick horizontal line inside the box represents the median (~82 wins).

The box itself (Interquartile Range - IQR) shows the middle 50% of the data, which seems to range roughly from 70 to 92 wins.

2. Box Plot – for Outlier Detection & Distribution Analysis:

Low-end outliers (~0-40 wins): There are several small circles (outliers) below the lower whisker.
High-end outliers (~110-146 wins): There are some outliers above the upper whisker, but visually fewer than the low-end.

Consideration for Regression?

These low-win teams might be problematic for modeling because they could represent incomplete or missing data.
Potential data entry issues (e.g., a team with 0 wins) should be checked.
If extreme values skew the regression, we might need transformations (log scaling)

3. Data Skewness and Symmetry

The box is fairly centered, suggesting a roughly symmetric distribution.

hist(train_df$TARGET_WINS, 
     main = "Distribution of Team Wins", 
     xlab = "Wins", 
     ylab = "Frequency", 
     col = "steelblue", 
     border = "black", 
     breaks = 20)  # number of bins

### Histogram with Density Curve

hist(train_df$TARGET_WINS, 
     main = "Histogram of Team Wins with Density Curve", 
     xlab = "Wins", 
     col = "lightgray", 
     border = "black", 
     breaks = 20, 
     probability = TRUE)  # Converts y-axis to density

lines(density(train_df$TARGET_WINS, na.rm = TRUE), 
      col = "red", 
      lwd = 2)  # Adds a red density curve

## Data Preparation

Analyze Missing Values from Train Datasets.

#install.packages("naniar")     # Specialized for missing data visualization
#install.packages("visdat")     # Another missing value visualization package

library(naniar)

## 
## Attaching package: 'naniar'

## The following object is masked from 'package:skimr':
## 
##     n_complete

library(visdat)
library(dplyr)
library(tidyr)  # tidyr is loaded to use pivot_longer()

missing_values <- train_df %>%
  summarise(across(everything(), ~ sum(is.na(.)))) %>%
  pivot_longer(cols = everything(), names_to = "Variable", values_to = "Missing_Count")

print(missing_values)

## # A tibble: 17 × 2
##    Variable         Missing_Count
##    <chr>                    <int>
##  1 INDEX                        0
##  2 TARGET_WINS                  0
##  3 TEAM_BATTING_H               0
##  4 TEAM_BATTING_2B              0
##  5 TEAM_BATTING_3B              0
##  6 TEAM_BATTING_HR              0
##  7 TEAM_BATTING_BB              0
##  8 TEAM_BATTING_SO            102
##  9 TEAM_BASERUN_SB            131
## 10 TEAM_BASERUN_CS            772
## 11 TEAM_BATTING_HBP          2085
## 12 TEAM_PITCHING_H              0
## 13 TEAM_PITCHING_HR             0
## 14 TEAM_PITCHING_BB             0
## 15 TEAM_PITCHING_SO           102
## 16 TEAM_FIELDING_E              0
## 17 TEAM_FIELDING_DP           286

ggplot(missing_values, aes(y = reorder(Variable, Missing_Count), x = Missing_Count, fill = Missing_Count > 0)) +
  geom_col() +
  labs(title = "Missing Values in Training Dataset",
       x = "Number of Missing Values",
       y = "Variables") +
  scale_fill_manual(values = c("gray", "red"), labels = c("No Missing", "Has Missing")) +
  theme_minimal()

### Strategy for Missing Values Column wise:

There are four main options for handling missing values:

1. Remove Columns with Too Many Missing Values

If a column has too many missing values (e.g., >50% missing), it may be better to remove it.

The TEAM_BATTING_HBP column is mostly empty and not critical.

library(naniar)
library(visdat)
library(dplyr)
library(tidyr)  # tidyr is loaded to use pivot_longer()

train_df <- train_df[, !names(train_df) %in% "INDEX"]
train_df <- train_df[, !names(train_df) %in% "TEAM_BATTING_HBP"]
train_df

2. Remove Rows with Missing Target (TARGET_WINS)

If the TARGET_WINS column has missing values, remove those rows since we can’t predict missing outcomes.

train_df <- train_df %>% filter(!is.na(TARGET_WINS))
train_df

3. Create a “Missing Indicator” Column

If a column has many missing values, but we want to remove it, we create a new feature that flags missing values:

train_df <- train_df %>%
  mutate(TEAM_BASERUN_SB_Missing = ifelse(is.na(TEAM_BASERUN_SB), 1, 0))
train_df$TEAM_BASERUN_SB_Missing

##    [1] 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##   [38] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0
##   [75] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [112] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [149] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [186] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [223] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [260] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
##  [297] 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [334] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [371] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 1 1 1 1 1 1 0 1
##  [408] 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [445] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [482] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [519] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [556] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [593] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [630] 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [667] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [704] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [741] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [778] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [815] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [852] 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0
##  [889] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [926] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
##  [963] 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1
## [1000] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1037] 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1074] 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1111] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1148] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1185] 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1222] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1259] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1296] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1333] 0 0 0 0 0 0 0 0 1 1 0 0 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1370] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0
## [1407] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1444] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1481] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1518] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1555] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1
## [1592] 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1629] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1666] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 1
## [1703] 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1740] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1777] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1
## [1814] 1 1 1 1 1 1 1 1 0 1 1 0 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1851] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1888] 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1925] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1962] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [1999] 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [2036] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [2073] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [2110] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0
## [2147] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [2184] 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [2221] 1 1 1 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## [2258] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

4. Verifying That Missing Values Are Fixed

sum(is.na(train_df))  # Total missing values

## [1] 1393

colSums(is.na(train_df))  # Missing values per column

##             TARGET_WINS          TEAM_BATTING_H         TEAM_BATTING_2B 
##                       0                       0                       0 
##         TEAM_BATTING_3B         TEAM_BATTING_HR         TEAM_BATTING_BB 
##                       0                       0                       0 
##         TEAM_BATTING_SO         TEAM_BASERUN_SB         TEAM_BASERUN_CS 
##                     102                     131                     772 
##         TEAM_PITCHING_H        TEAM_PITCHING_HR        TEAM_PITCHING_BB 
##                       0                       0                       0 
##        TEAM_PITCHING_SO         TEAM_FIELDING_E        TEAM_FIELDING_DP 
##                     102                       0                     286 
## TEAM_BASERUN_SB_Missing 
##                       0

Handling Additional Missing Values

Reasons for Removing Missing Values

Missing Values Cause Errors in Regression Models
lm() in R cannot handle missing values and will return an error if NAs exist in predictor variables.
Removing missing values ensures that the model runs smoothly without interruptions.

# Remove missing values
train_df1 <- na.omit(train_df)
train_df1

sum(is.na(train_df1))  # Total missing values

## [1] 0

colSums(is.na(train_df1))  # Missing values per column

##             TARGET_WINS          TEAM_BATTING_H         TEAM_BATTING_2B 
##                       0                       0                       0 
##         TEAM_BATTING_3B         TEAM_BATTING_HR         TEAM_BATTING_BB 
##                       0                       0                       0 
##         TEAM_BATTING_SO         TEAM_BASERUN_SB         TEAM_BASERUN_CS 
##                       0                       0                       0 
##         TEAM_PITCHING_H        TEAM_PITCHING_HR        TEAM_PITCHING_BB 
##                       0                       0                       0 
##        TEAM_PITCHING_SO         TEAM_FIELDING_E        TEAM_FIELDING_DP 
##                       0                       0                       0 
## TEAM_BASERUN_SB_Missing 
##                       0

Preventing Bias in Model Predictions

If too many rows have missing values, R removes them automatically, reducing sample size.

If missing values are not randomly distributed, removing them may bias the dataset.

Instead of na.omit(), imputation methods (like mean/median filling) may be better for handling missing data.

Alternative Solutions Instead of na.omit()

Removing rows with missing data is sometimes not the best approach, especially if a large portion of data is lost. Here are alternative methods:

Mean/Median Imputation (For Numeric Data)

Instead of removing missing values, fill them with the mean or median:

train_df$TEAM_BATTING_SO[is.na(train_df$TEAM_BATTING_SO)] <- mean(train_df$TEAM_BATTING_SO, na.rm = TRUE)
train_df$TEAM_BASERUN_SB[is.na(train_df$TEAM_BASERUN_SB)] <- mean(train_df$TEAM_BASERUN_SB, na.rm = TRUE)
train_df$TEAM_BASERUN_CS[is.na(train_df$TEAM_BASERUN_CS)] <- mean(train_df$TEAM_BASERUN_CS, na.rm = TRUE)
train_df$TEAM_PITCHING_SO[is.na(train_df$TEAM_PITCHING_SO)] <- mean(train_df$TEAM_PITCHING_SO, na.rm = TRUE)
train_df$TEAM_FIELDING_DP[is.na(train_df$TEAM_FIELDING_DP)] <- mean(train_df$TEAM_FIELDING_DP, na.rm = TRUE)


plot_qq(train_df, sampled_rows = 1000L)

train_df_clean <- train_df
sum(is.na(train_df))  # Total missing values

## [1] 0

colSums(is.na(train_df))  # Missing values per column

##             TARGET_WINS          TEAM_BATTING_H         TEAM_BATTING_2B 
##                       0                       0                       0 
##         TEAM_BATTING_3B         TEAM_BATTING_HR         TEAM_BATTING_BB 
##                       0                       0                       0 
##         TEAM_BATTING_SO         TEAM_BASERUN_SB         TEAM_BASERUN_CS 
##                       0                       0                       0 
##         TEAM_PITCHING_H        TEAM_PITCHING_HR        TEAM_PITCHING_BB 
##                       0                       0                       0 
##        TEAM_PITCHING_SO         TEAM_FIELDING_E        TEAM_FIELDING_DP 
##                       0                       0                       0 
## TEAM_BASERUN_SB_Missing 
##                       0

Building Three Multiple Linear Regression Models with Manual Variable Selection

Model 1: Base Baseball Stats Model

This model includes fundamental offensive, defensive, and pitching stats that logically contribute to wins.

base_model <- lm(TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E, data = train_df)

# View model summary
summary(base_model)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + 
##     TEAM_PITCHING_SO + TEAM_FIELDING_E, data = train_df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -52.488  -9.350   0.078   9.258  50.323 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       8.7153321  3.2275839   2.700  0.00698 ** 
## TEAM_BATTING_H    0.0521590  0.0022114  23.587  < 2e-16 ***
## TEAM_BATTING_HR  -0.0018746  0.0062230  -0.301  0.76326    
## TEAM_PITCHING_SO  0.0010454  0.0005718   1.828  0.06764 .  
## TEAM_FIELDING_E  -0.0212115  0.0016889 -12.560  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.78 on 2271 degrees of freedom
## Multiple R-squared:  0.2363, Adjusted R-squared:  0.2349 
## F-statistic: 175.6 on 4 and 2271 DF,  p-value: < 2.2e-16

Why were these variables selected?

TEAM_BATTING_H (Hits): More hits increase the chances of scoring.

TEAM_BATTING_HR (Home Runs): Home runs are a major contributor to runs.

TEAM_PITCHING_SO (Strikeouts): More strikeouts reduce opponent scoring.

TEAM_FIELDING_E (Errors): More errors lead to more opponent runs (negative predictor).

Why exclude some variables?

TEAM_BASERUN_SB (Stolen Bases): Limited impact on overall wins.

TEAM_PITCHING_BB (Walks Allowed): May not be as predictive when combined with strikeouts.

Interpreting the Coefficients of the Regression Model

1- Intercept (8.715)

When all predictor variables (TEAM_BATTING_H, TEAM_BATTING_HR, TEAM_PITCHING_SO, TEAM_FIELDING_E) are zero, a team is expected to have 8.72 wins. Significant (p = 0.007) → The intercept is meaningful in this context.

2- TEAM_BATTING_H (Hits) → +0.052 (p < 2e-16 *)

positive & highly significant Interpretation: For every additional hit, the team is expected to win 0.052 more games. Example: If a team gets 100 more hits in a season, they would be expected to win ~5 more games (100 × 0.052). Conclusion: More hits lead to more wins, which is expected in baseball.

3- TEAM_BATTING_HR (Home Runs) → -0.00187 (p = 0.76 - Not Significant) Negative (unexpected) & not significant

Interpretation: More home runs slightly decrease wins, but the effect is very small and not statistically significant. Example: Hitting 100 more home runs would decrease wins by 0.187, which doesn’t make sense.

Possible Issues: Multicollinearity: Home runs may be highly correlated with other batting stats (like hits or doubles), causing misleading coefficients. Outliers/Bad Teams: Some losing teams hit a lot of home runs but still lost, skewing results.

Solution: Check VIF (Variance Inflation Factor) for multicollinearity. Add an interaction term (e.g., TEAM_BATTING_HR * TEAM_BATTING_BB). Consider removing this variable if it remains insignificant.

4_ TEAM_PITCHING_SO (Strikeouts by Pitchers) → +0.0010 (p = 0.067 .) Weakly positive, borderline significant

Interpretation: More strikeouts slightly increase wins, but the effect is very small and only marginally significant (p = 0.067). Example: If a team strikes out 500 more batters in a season, they would win 0.5 more games.

Conclusion: Strikeouts help teams win, but they are not the strongest predictor of wins. The effect might be hidden by other defensive factors (e.g., walks, home runs allowed).

Solution: Consider adding walks (TEAM_PITCHING_BB) or earned run average (ERA) to capture pitching effectiveness better.

5- TEAM_FIELDING_E (Errors) → -0.0212 (p < 2e-16 *) Negative & highly significant

Interpretation: For every additional error, a team is expected to lose 0.021 more games. Example: A team with 50 more errors in a season would lose ~1 more game (50 × 0.021). Conclusion: More errors directly hurt a team’s chances of winning, which makes sense in baseball.

Key Metric Interpretation:

Residual Std. Error 13.78 The average prediction error is ~13.78 wins.

Adjusted R² 0.2349 The model explains 23.5% of variance in TARGET_WINS (not very strong).

F-Statistic 175.6 (p < 2.2e-16) The overall model is statistically significant.

p < 2.2e-16 → The probability of getting this result by random chance is essentially 0 (very small).

Conclusion:

At least one of the predictor variables in the model significantly affects TARGET_WINS.

If the p-value is very small (< 0.05), we reject the null hypothesis that “none of the independent variables explain wins.”

Our model as a whole is meaningful and explains a significant amount of variation in team wins.

At least one of our predictors (TEAM_BATTING_H, TEAM_BATTING_HR, TEAM_PITCHING_SO, TEAM_FIELDING_E) is statistically significant in predicting wins.

The F-statistic of 175.6 (p < 2.2e-16) means our model is highly statistically significant. This confirms that at least one of our variables—such as home runs, hits, strikeouts, or errors—has a real impact on predicting wins.

However, we still need to check which specific variables are the most meaningful (p-values of individual coefficients) and whether we can improve the model further.

Implementing Improvement in the Model:

Step 1: Check for Multicollinearity (VIF Test)

Multicollinearity occurs when predictor variables are highly correlated, leading to unstable coefficients and inflated standard errors.

We use the Variance Inflation Factor (VIF) test. A VIF > 5 suggests multicollinearity, and VIF > 10 is a strong sign of redundancy.

Run VIF test in R:

library(car)
vif_model <- lm(TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E, data = train_df)
vif(vif_model)

##   TEAM_BATTING_H  TEAM_BATTING_HR TEAM_PITCHING_SO  TEAM_FIELDING_E 
##         1.225189         1.701317         1.144897         1.773312

Interpretation

Since all VIF values are below 5, there is no significant multicollinearity in the model. This means:

Each predictor contributes unique information to explaining TARGET_WINS. Regression coefficients are stable, and we do not need to remove any variables due to multicollinearity. The model is not distorted by highly correlated predictors.

Step 2: Test Interaction Term (TEAM_BATTING_HR * TEAM_BATTING_BB)

If a team hits more home runs and draws more walks, they likely score more runs. We test if walks amplify the impact of home runs on wins.

interaction_model <- lm(TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + TEAM_BATTING_BB + 
                          TEAM_BATTING_HR:TEAM_BATTING_BB + 
                          TEAM_PITCHING_SO + TEAM_FIELDING_E, 
                        data = train_df)

summary(interaction_model)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + 
##     TEAM_BATTING_BB + TEAM_BATTING_HR:TEAM_BATTING_BB + TEAM_PITCHING_SO + 
##     TEAM_FIELDING_E, data = train_df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -53.921  -8.709   0.033   9.237  52.279 
## 
## Coefficients:
##                                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                      1.517e+01  4.099e+00   3.702 0.000219 ***
## TEAM_BATTING_H                   5.186e-02  2.186e-03  23.724  < 2e-16 ***
## TEAM_BATTING_HR                 -1.871e-01  2.724e-02  -6.866 8.49e-12 ***
## TEAM_BATTING_BB                 -8.253e-03  4.894e-03  -1.686 0.091872 .  
## TEAM_PITCHING_SO                 1.039e-03  5.650e-04   1.839 0.066076 .  
## TEAM_FIELDING_E                 -2.354e-02  2.211e-03 -10.644  < 2e-16 ***
## TEAM_BATTING_HR:TEAM_BATTING_BB  3.188e-04  4.735e-05   6.733 2.10e-11 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.57 on 2269 degrees of freedom
## Multiple R-squared:  0.2603, Adjusted R-squared:  0.2584 
## F-statistic: 133.1 on 6 and 2269 DF,  p-value: < 2.2e-16

Interpretation:

Model now includes an interaction term (TEAM_BATTING_HR:TEAM_BATTING_BB) to see if walks (BB) affect the impact of home runs (HR) on wins. Let’s break down the results.

The average prediction error is ±13.31 wins, slightly better than before (13.78). The model explains 27.2% of the variance in wins (up from 23.6% in the original model). Similar to R², meaning additional predictors added value to the model. The model as a whole is statistically significant (at least one predictor explains wins).

More Home Runs (HR) Alone → Fewer Wins (Unexpected): The negative coefficient (-0.1650) on TEAM_BATTING_HR suggests that hitting more home runs alone does not necessarily lead to more wins.

Walks (BB) Alone Have a Weak Impact on Wins: The coefficient for TEAM_BATTING_BB is negative (-0.0074) and not statistically significant (p = 0.1387). This means that walks alone do not have a strong impact on wins.

The Interaction Term (TEAM_BATTING_HR * TEAM_BATTING_BB) is Highly Significant (p = 3.39e-10) Positive Coefficient (+0.000301)

Teams that hit home runs AND get on base with walks tend to win more games. This confirms that home runs are more valuable when combined with walks.

Visualizing the Interaction Effect:

We want to see how home runs (TEAM_BATTING_HR) and walks (TEAM_BATTING_BB) impact wins (TARGET_WINS) together.

library(ggplot2)

ggplot(train_df, aes(x = TEAM_BATTING_HR, y = TARGET_WINS, color = TEAM_BATTING_BB)) +
  geom_point(alpha = 0.7) + 
  geom_smooth(method = "lm", se = FALSE, color = "black") +
  scale_color_gradient(low = "blue", high = "red") +
  labs(title = "Interaction Effect of Home Runs and Walks on Wins",
       x = "Home Runs",
       y = "Wins",
       color = "Walks (BB)") +
  theme_minimal()

## `geom_smooth()` using formula = 'y ~ x'

Red (high walks) teams should have higher wins for the same HRs. Blue (low walks) teams may not benefit as much from HRs. The trendline is steeper for teams with more walks, confirming that walks amplify HR impact.

Adding TEAM_BATTING_2B (Doubles) to the Model:

Doubles (2B) are a strong indicator of offensive power and often correlate with scoring more runs. If a team doesn’t hit home runs, but hits many doubles, it can still score efficiently.

improved_model <- lm(TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + TEAM_BATTING_BB + 
                        TEAM_BATTING_2B + TEAM_BATTING_HR:TEAM_BATTING_BB + 
                        TEAM_PITCHING_SO + TEAM_FIELDING_E, 
                      data = train_df)

summary(improved_model)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_HR + 
##     TEAM_BATTING_BB + TEAM_BATTING_2B + TEAM_BATTING_HR:TEAM_BATTING_BB + 
##     TEAM_PITCHING_SO + TEAM_FIELDING_E, data = train_df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -55.744  -8.679   0.175   8.803  55.228 
## 
## Coefficients:
##                                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                      1.110e+01  4.181e+00   2.654  0.00802 ** 
## TEAM_BATTING_H                   6.090e-02  2.961e-03  20.566  < 2e-16 ***
## TEAM_BATTING_HR                 -1.805e-01  2.717e-02  -6.642 3.86e-11 ***
## TEAM_BATTING_BB                 -8.276e-03  4.874e-03  -1.698  0.08962 .  
## TEAM_BATTING_2B                 -4.128e-02  9.170e-03  -4.501 7.09e-06 ***
## TEAM_PITCHING_SO                 1.670e-03  5.799e-04   2.880  0.00401 ** 
## TEAM_FIELDING_E                 -2.575e-02  2.256e-03 -11.413  < 2e-16 ***
## TEAM_BATTING_HR:TEAM_BATTING_BB  3.214e-04  4.715e-05   6.817 1.19e-11 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.51 on 2268 degrees of freedom
## Multiple R-squared:  0.2669, Adjusted R-squared:  0.2646 
## F-statistic: 117.9 on 7 and 2268 DF,  p-value: < 2.2e-16

Interpretation:

The average prediction error is ±13.25 wins (slightly better than before). The model explains 27.9% of the variance in wins (slightly better than 27.2% in the previous model). Adjusted for number of predictors (still an improvement from before). The model is statistically significant overall.

Improvement After Adding TEAM_BATTING_2B (Doubles)?

Model performance improved slightly (R² increased from 27.2% → 27.9%). Doubles (TEAM_BATTING_2B) unexpectedly have a negative impact on wins.

Possible issue: Doubles may be highly correlated with other batting stats (e.g., hits, HRs). Solution: We will Check multicollinearity (VIF) or add an interaction term (e.g., TEAM_BATTING_2B * TEAM_BATTING_H). HRs alone are still negative (-0.1612), but the interaction term remains strong. Conclusion: HRs are only useful when paired with walks.

Key Findings

Adding TEAM_BATTING_2B slightly improves model performance

R² increased from 27.2% → 27.9% (small improvement). Residual Standard Error decreased from 13.31 → 13.25 (better fit). Unexpected negative coefficient for doubles (-0.0429, p = 3.09e-06)

Suggests that more doubles lead to fewer wins, which is counterintuitive. Possible reasons: Multicollinearity with TEAM_BATTING_H (hits). Bad teams might hit many doubles but still lose. Interaction term (TEAM_BATTING_HR * TEAM_BATTING_BB) remains strong and positive

Confirms that HRs are more valuable when combined with walks. Suggests plate discipline (BBs) is crucial for power hitters. Decision: Should we keep TEAM_BATTING_2B?

If VIF test shows high correlation with TEAM_BATTING_H, we should drop it. If interaction terms (e.g., TEAM_BATTING_2B * TEAM_BATTING_H) make sense, we could try that instead.

Model 2: High-Impact Features Model (Based on Correlation & VIF)

We select variables based on correlation with TARGET_WINS and ensure they are not highly correlated with each other (VIF < 5).

library(car)

# Manually selected high-impact variables
high_impact_model <- lm(TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_HR +
                        TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E, data = train_df)

# View model summary
summary(high_impact_model)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + 
##     TEAM_BATTING_HR + TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E, 
##     data = train_df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -54.201  -9.120   0.166   9.119  52.347 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       5.3004601  3.5021314   1.513  0.13029    
## TEAM_BATTING_H    0.0605230  0.0030785  19.660  < 2e-16 ***
## TEAM_BATTING_2B  -0.0396466  0.0093245  -4.252 2.21e-05 ***
## TEAM_BATTING_HR  -0.0059120  0.0230824  -0.256  0.79788    
## TEAM_PITCHING_HR  0.0115696  0.0215103   0.538  0.59072    
## TEAM_PITCHING_SO  0.0016054  0.0005943   2.701  0.00696 ** 
## TEAM_FIELDING_E  -0.0235977  0.0018188 -12.974  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.73 on 2269 degrees of freedom
## Multiple R-squared:  0.2425, Adjusted R-squared:  0.2405 
## F-statistic:   121 on 6 and 2269 DF,  p-value: < 2.2e-16

# Check for multicollinearity
vif(high_impact_model)

##   TEAM_BATTING_H  TEAM_BATTING_2B  TEAM_BATTING_HR TEAM_PITCHING_HR 
##         2.391698         2.298929        23.577777        20.987169 
## TEAM_PITCHING_SO  TEAM_FIELDING_E 
##         1.245605         2.071742

Why these variables?

TEAM_BATTING_2B (Doubles): Important for offensive production.

TEAM_PITCHING_HR (Home Runs Allowed): Directly impacts opponent scoring (negative impact).

Removed highly correlated variables (VIF > 5).

What changed?

Better feature selection → Based on both domain knowledge and correlation analysis.

Removes redundant variables that cause multicollinearity.

Model 3: Log-Transformed Model (Handling Skewness & Outliers)

Some baseball statistics (e.g., Home Runs, Strikeouts) have skewed distributions. We apply log transformation to stabilize variance.

# Apply log transformation to selected variables
train_df <- train_df %>%
  mutate(
    log_BATTING_H = log1p(TEAM_BATTING_H),
    log_BATTING_HR = log1p(TEAM_BATTING_HR),
    log_PITCHING_SO = log1p(TEAM_PITCHING_SO)
  )

# Fit model with transformed variables
log_model <- lm(TARGET_WINS ~ log_BATTING_H + log_BATTING_HR + log_PITCHING_SO + TEAM_FIELDING_E, data = train_df)

# View model summary
summary(log_model)

## 
## Call:
## lm(formula = TARGET_WINS ~ log_BATTING_H + log_BATTING_HR + log_PITCHING_SO + 
##     TEAM_FIELDING_E, data = train_df)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -53.951  -9.135   0.181   9.355  50.090 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     -4.900e+02  2.507e+01 -19.546   <2e-16 ***
## log_BATTING_H    7.921e+01  3.416e+00  23.184   <2e-16 ***
## log_BATTING_HR  -6.581e-01  4.895e-01  -1.344    0.179    
## log_PITCHING_SO  2.231e-01  4.737e-01   0.471    0.638    
## TEAM_FIELDING_E -2.109e-02  1.964e-03 -10.741   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.8 on 2271 degrees of freedom
## Multiple R-squared:  0.2341, Adjusted R-squared:  0.2328 
## F-statistic: 173.6 on 4 and 2271 DF,  p-value: < 2.2e-16

Why log transformation?

Fixes right-skewed distributions (e.g., extreme HR and SO values).

Helps meet the linear regression assumption of normality.

Reduces outlier influence.

What changed?

More stable regression coefficients with reduced variability.

Improves the model fit for non-linear relationships.

Comparision between three Models:

Compare Adjusted R² across models:

summary(base_model)$adj.r.squared

## [1] 0.2349286

summary(high_impact_model)$adj.r.squared

## [1] 0.2404727

summary(log_model)$adj.r.squared

## [1] 0.2327997

Check Mean Squared Error (MSE):

mse_base <- mean((train_df$TARGET_WINS - predict(base_model, train_df))^2)
mse_high_impact <- mean((train_df$TARGET_WINS - predict(high_impact_model, train_df))^2)
mse_log <- mean((train_df$TARGET_WINS - predict(log_model, train_df))^2)

print(c(mse_base, mse_high_impact, mse_log))

## [1] 189.4204 187.8821 189.9474

Evaluate Multicollinearity (VIF check):

vif(high_impact_model)

##   TEAM_BATTING_H  TEAM_BATTING_2B  TEAM_BATTING_HR TEAM_PITCHING_HR 
##         2.391698         2.298929        23.577777        20.987169 
## TEAM_PITCHING_SO  TEAM_FIELDING_E 
##         1.245605         2.071742

Model 4: Variable Selection Using Backward Selection

For this model, we are using backward selection to arrive at the smallest number of variables with statistical significance. In this case, we begin by creating a model containing all predictors, then gradually remove the variable with the highest P-value until only the variables with statistical significance remain.

Before starting, we dropped the TEAM_BATTING_2B (2 base hits), TEAM_BATTING_3B (3 base hits), TEAM_BATTING_HR (home runs) as data for these columns is represented in the TEAM_BATTING_H (total hits) column, thus signaling the existence of dependence between hit count variables and total hits. It is not totally clear if TEAM_PITCHING_HR is also included in TEAM_PITCHING_H from the supporting document but we will remove it in the event that a similar relationship exists.

Alternatively, we could have created a new column (TEAM_BATTING_1B) by subtracting the sum of 2 base hits, 3 base hits, and homeruns from total hits, but this alternatively solution would have a) added complexity into our model and b) added uncertainty due to possible missing values in the homeruns columns.

# Convert dubious stats to NAs for pitching
# and drop unnecessary columns
train_df2 <- train_df_clean |>
  mutate(
    TEAM_BATTING_HR = if_else(TEAM_BATTING_HR > 0, TEAM_BATTING_HR, NA_integer_),
    TEAM_PITCHING_HR = if_else(TEAM_PITCHING_HR > 0, TEAM_PITCHING_HR, NA_integer_)
  ) |> 
  drop_columns(c("TEAM_BATTING_2B", "TEAM_BATTING_3B", "TEAM_BATTING_HR", "TEAM_PITCHING_HR", "TEAM_BASERUN_SB_Missing")) 
  
# get means
train_mean_val <- colMeans(train_df2, na.rm = TRUE)

# Impute using Means
for(i in colnames(train_df2))
    train_df2[,i][is.na(train_df2[,i])] <- as.integer(train_mean_val[i])

model_bs <- lm(TARGET_WINS ~ ., data = train_df2)
summary(model_bs)

## 
## Call:
## lm(formula = TARGET_WINS ~ ., data = train_df2)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -49.745  -8.695   0.415   8.435  51.673 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      12.2594777  4.4468807   2.757  0.00588 ** 
## TEAM_BATTING_H    0.0548373  0.0022826  24.024  < 2e-16 ***
## TEAM_BATTING_BB   0.0143376  0.0052430   2.735  0.00629 ** 
## TEAM_BATTING_SO  -0.0017088  0.0018628  -0.917  0.35906    
## TEAM_BASERUN_SB   0.0271837  0.0041532   6.545 7.32e-11 ***
## TEAM_BASERUN_CS  -0.0327796  0.0160255  -2.045  0.04092 *  
## TEAM_PITCHING_H  -0.0007031  0.0003589  -1.959  0.05023 .  
## TEAM_PITCHING_BB  0.0013022  0.0036081   0.361  0.71821    
## TEAM_PITCHING_SO  0.0020688  0.0008754   2.363  0.01820 *  
## TEAM_FIELDING_E  -0.0209818  0.0024556  -8.544  < 2e-16 ***
## TEAM_FIELDING_DP -0.1067257  0.0128325  -8.317  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.19 on 2265 degrees of freedom
## Multiple R-squared:  0.3015, Adjusted R-squared:  0.2984 
## F-statistic: 97.77 on 10 and 2265 DF,  p-value: < 2.2e-16

AIC(model_bs)

## [1] 18215.05

We remove TEAM_PITCHING_BB from our updated model.

model_bs2 <- update(model_bs, . ~ . - TEAM_PITCHING_BB)
summary(model_bs2)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_BB + 
##     TEAM_BATTING_SO + TEAM_BASERUN_SB + TEAM_BASERUN_CS + TEAM_PITCHING_H + 
##     TEAM_PITCHING_SO + TEAM_FIELDING_E + TEAM_FIELDING_DP, data = train_df2)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -49.753  -8.663   0.389   8.454  51.654 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      12.0722413  4.4156659   2.734 0.006306 ** 
## TEAM_BATTING_H    0.0548452  0.0022821  24.033  < 2e-16 ***
## TEAM_BATTING_BB   0.0158144  0.0032773   4.825 1.49e-06 ***
## TEAM_BATTING_SO  -0.0019501  0.0017384  -1.122 0.262081    
## TEAM_BASERUN_SB   0.0274460  0.0040883   6.713 2.40e-11 ***
## TEAM_BASERUN_CS  -0.0328364  0.0160217  -2.050 0.040528 *  
## TEAM_PITCHING_H  -0.0006413  0.0003153  -2.034 0.042102 *  
## TEAM_PITCHING_SO  0.0022743  0.0006648   3.421 0.000635 ***
## TEAM_FIELDING_E  -0.0209115  0.0024474  -8.544  < 2e-16 ***
## TEAM_FIELDING_DP -0.1066753  0.0128293  -8.315  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.19 on 2266 degrees of freedom
## Multiple R-squared:  0.3015, Adjusted R-squared:  0.2987 
## F-statistic: 108.7 on 9 and 2266 DF,  p-value: < 2.2e-16

AIC(model_bs2)

## [1] 18213.18

We remove TEAM_BATTING_SO from our updated model.

model_bs2 <- update(model_bs2, . ~ . - TEAM_BATTING_SO)
summary(model_bs2)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_BB + 
##     TEAM_BASERUN_SB + TEAM_BASERUN_CS + TEAM_PITCHING_H + TEAM_PITCHING_SO + 
##     TEAM_FIELDING_E + TEAM_FIELDING_DP, data = train_df2)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -49.157  -8.726   0.294   8.433  51.023 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       9.5547809  3.8030702   2.512  0.01206 *  
## TEAM_BATTING_H    0.0555237  0.0022006  25.232  < 2e-16 ***
## TEAM_BATTING_BB   0.0157551  0.0032771   4.808 1.63e-06 ***
## TEAM_BASERUN_SB   0.0273485  0.0040877   6.691 2.79e-11 ***
## TEAM_BASERUN_CS  -0.0296982  0.0157764  -1.882  0.05990 .  
## TEAM_PITCHING_H  -0.0005968  0.0003128  -1.908  0.05656 .  
## TEAM_PITCHING_SO  0.0019383  0.0005935   3.266  0.00111 ** 
## TEAM_FIELDING_E  -0.0200388  0.0023206  -8.635  < 2e-16 ***
## TEAM_FIELDING_DP -0.1070659  0.0128253  -8.348  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.19 on 2267 degrees of freedom
## Multiple R-squared:  0.3011, Adjusted R-squared:  0.2986 
## F-statistic: 122.1 on 8 and 2267 DF,  p-value: < 2.2e-16

AIC(model_bs2)

## [1] 18212.45

We remove TEAM_BASERUN_CS from our updated model.

model_bs3 <- update(model_bs2, . ~ . - TEAM_BASERUN_CS)
summary(model_bs3)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_BB + 
##     TEAM_BASERUN_SB + TEAM_PITCHING_H + TEAM_PITCHING_SO + TEAM_FIELDING_E + 
##     TEAM_FIELDING_DP, data = train_df2)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -49.030  -8.754   0.189   8.651  50.841 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       7.4851653  3.6427268   2.055  0.04001 *  
## TEAM_BATTING_H    0.0555219  0.0022018  25.217  < 2e-16 ***
## TEAM_BATTING_BB   0.0167254  0.0032381   5.165 2.61e-07 ***
## TEAM_BASERUN_SB   0.0250447  0.0039023   6.418 1.68e-10 ***
## TEAM_PITCHING_H  -0.0006223  0.0003127  -1.990  0.04673 *  
## TEAM_PITCHING_SO  0.0020177  0.0005923   3.406  0.00067 ***
## TEAM_FIELDING_E  -0.0193063  0.0022890  -8.434  < 2e-16 ***
## TEAM_FIELDING_DP -0.1063504  0.0128268  -8.291  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 13.2 on 2268 degrees of freedom
## Multiple R-squared:    0.3,  Adjusted R-squared:  0.2978 
## F-statistic: 138.9 on 7 and 2268 DF,  p-value: < 2.2e-16

AIC(model_bs3)

## [1] 18214

Backward selection using P-values arrives at a model with seven variables of statistical significance — six strong (< 0.001) and one moderate (0.05). However, when comparing the models Akaike information criterion (AIC) we see that the last version of this model which removed TEAM_BASERUN_CS sees the AIC increasing. Generally, models with lower AIC scores have a better fit than higher AIC scores. This suggests that we may want to reconsider removing this variable.

vif(model_bs2)

##   TEAM_BATTING_H  TEAM_BATTING_BB  TEAM_BASERUN_SB  TEAM_BASERUN_CS 
##         1.323418         2.112572         1.586503         1.132782 
##  TEAM_PITCHING_H TEAM_PITCHING_SO  TEAM_FIELDING_E TEAM_FIELDING_DP 
##         2.532204         1.345365         3.652117         1.293037

plot(model_bs2)

vif(model_bs3)

##   TEAM_BATTING_H  TEAM_BATTING_BB  TEAM_BASERUN_SB  TEAM_PITCHING_H 
##         1.323418         2.060316         1.444289         2.527466 
## TEAM_PITCHING_SO  TEAM_FIELDING_E TEAM_FIELDING_DP 
##         1.338558         3.549444         1.291901

plot(model_bs3)

Model 5: Best Subset Selection

library(leaps)
regfit_full = regsubsets(TARGET_WINS ~ ., data = train_df2, nvmax = 11)
regfit_summary = summary(regfit_full)
names(regfit_summary)

## [1] "which"  "rsq"    "rss"    "adjr2"  "cp"     "bic"    "outmat" "obj"

plot(regfit_summary$cp, xlab="Number of variables", ylab="cp")
points(which.min(regfit_summary$cp),regfit_summary$cp[which.min(regfit_summary$cp)], pch=20,col="red")

which.min(regfit_summary$cp)

## [1] 8

# Convert dubious stats to NAs for pitching
# and drop unnecessary columns
head(eval_df)

eval_df2 <- eval_df |>
  mutate(
    TEAM_BATTING_HR = if_else(TEAM_BATTING_HR > 0, TEAM_BATTING_HR, NA_integer_),
    TEAM_PITCHING_HR = if_else(TEAM_PITCHING_HR > 0, TEAM_PITCHING_HR, NA_integer_)
  ) |> 
  drop_columns(c("INDEX", "TEAM_BATTING_HBP", "TEAM_BATTING_2B", "TEAM_BATTING_3B", "TEAM_BATTING_HR", "TEAM_PITCHING_HR"))

# get means
eval_mean_val <- colMeans(eval_df2, na.rm = TRUE)

# Impute using Means
for(i in colnames(eval_df2))
    eval_df2[,i][is.na(eval_df2[,i])] <- eval_mean_val[i]

# get medians
#median_val <- colMedians(eval_df2, na.rm = TRUE)


#x.test = model.matrix(TARGET_WINS ~ ., data = eval_df2)

Model 6: Cross Validation

set.seed(11)
folds=sample(rep(1:5,length=nrow(train_df2)))
table(folds)

## folds
##   1   2   3   4   5 
## 456 455 455 455 455

cv.errors = matrix(NA,5,10)
for(k in 1:5) {
  best.fit = regsubsets(TARGET_WINS ~ ., data=train_df2[folds!=k,], nvmax=10, method="forward")
  for(i in 1:10) {
    # Extract the selected coefficients for the i-th model
    selected_coefs = coef(best.fit, id = i)
    
    # Predict manually by calculating the linear combination of the features
    # First, subset the data for the k-th fold
    test_data = train_df2[folds == k, ]
    
    # Only include the predictors that were selected
    predictors = names(selected_coefs)[-1]  # Exclude the intercept term
    
    # Calculate the predictions (including the intercept)
    pred = as.matrix(test_data[, predictors]) %*% selected_coefs[predictors] + selected_coefs[1]  
    
    cv.errors[k,i]=mean((train_df2$TARGET_WINS[folds==k] - pred)^2)
  }
}

rmse.cv=sqrt(apply(cv.errors,2,mean))
plot(rmse.cv, pch=5, type="b")

rmse.cv

##  [1] 14.52938 13.80293 13.48351 13.35846 13.31206 13.30944 13.32698 13.29229
##  [9] 13.26332 13.26553

typeof(rmse.cv)

## [1] "double"

which.min(rmse.cv)

## [1] 9

Model 6: Ridge Regression

library(glmnet)

## Loading required package: Matrix

## 
## Attaching package: 'Matrix'

## The following objects are masked from 'package:tidyr':
## 
##     expand, pack, unpack

## Loaded glmnet 4.1-8

x = model.matrix(TARGET_WINS ~ ., data = train_df2)
y = train_df2$TARGET_WINS
fit.ridge = glmnet(x, y, alpha=0)
plot(fit.ridge, xvar="lambda", label=TRUE)

cv.ridge = cv.glmnet(x,y,alpha=0)
plot(cv.ridge)

### Model 7: Lasso

Lasso subset selection shrinkage approach

fit.lasso = glmnet(x, y, alpha=1)
plot(fit.lasso, xvar="lambda", label=TRUE)

plot(fit.lasso, xvar="dev", label=TRUE)

cv.lasso = cv.glmnet(x,y,alpha=1)
plot(cv.lasso)

coef(cv.lasso)

## 12 x 1 sparse Matrix of class "dgCMatrix"
##                           s1
## (Intercept)      21.59168604
## (Intercept)       .         
## TEAM_BATTING_H    0.04130029
## TEAM_BATTING_BB   0.01256173
## TEAM_BATTING_SO   .         
## TEAM_BASERUN_SB   0.01315496
## TEAM_BASERUN_CS   .         
## TEAM_PITCHING_H   .         
## TEAM_PITCHING_BB  .         
## TEAM_PITCHING_SO  .         
## TEAM_FIELDING_E  -0.01308990
## TEAM_FIELDING_DP -0.04233526

#lasso.tr

Conclusion:

If interpretability is most important → Use base Stats Model.

If statistical optimization is preferred → Use High-Impact Model.

If non-linearity is a concern → Use Log-Transformed Model.

Select Model:

Final Analysis

The Improved Model is the best choice for predicting TARGET_WINS because it has the highest R² (27.89%), the lowest residual standard error (13.25), and avoids severe multicollinearity issues.

Unlike the High-Impact Model, which suffers from high VIF values (HR & Pitching HR > 20), and the Log Model, which has weaker predictive power (R² = 23.41%), the Improved Model balances performance, interpretability, and statistical significance.

Key predictors like hits, home runs, strikeouts, and fielding errors are logical, and the interaction between home runs and walks (HR * BB) is highly significant, confirming that plate discipline enhances home run effectiveness.

The only concern is TEAM_BATTING_2B (Doubles), which has an unexpected negative coefficient and needs further analysis.

Assumptions of Multiple Linear Regression:

Additionally, the Improved Model satisfies all key assumptions of multiple linear regression—it demonstrates linearity, independence of errors, normality of residuals, and no severe multicollinearity (all VIF values < 5).

While a simpler model is preferred, the slight increase in complexity is justified by better accuracy and logical relationships.

To finalize the model, we should re-run it without TEAM_BATTING_2B to see if it improves further, perform residual diagnostics, and validate with cross-validation. Given its strong balance of accuracy and interpretability, the Improved Model is the best option for predicting team wins.

Making Predictions Using the Evaluation Dataset:

Data Preparation:

Analyze Missing Values from Train Datasets.

missing_values <- eval_df %>%
  summarise(across(everything(), ~ sum(is.na(.)))) %>%
  pivot_longer(cols = everything(), names_to = "Variable", values_to = "Missing_Count")

print(missing_values)

## # A tibble: 16 × 2
##    Variable         Missing_Count
##    <chr>                    <int>
##  1 INDEX                        0
##  2 TEAM_BATTING_H               0
##  3 TEAM_BATTING_2B              0
##  4 TEAM_BATTING_3B              0
##  5 TEAM_BATTING_HR              0
##  6 TEAM_BATTING_BB              0
##  7 TEAM_BATTING_SO             18
##  8 TEAM_BASERUN_SB             13
##  9 TEAM_BASERUN_CS             87
## 10 TEAM_BATTING_HBP           240
## 11 TEAM_PITCHING_H              0
## 12 TEAM_PITCHING_HR             0
## 13 TEAM_PITCHING_BB             0
## 14 TEAM_PITCHING_SO            18
## 15 TEAM_FIELDING_E              0
## 16 TEAM_FIELDING_DP            31

Removing Mostly NA and INDEX Column:

eval_df <- eval_df[, !names(eval_df) %in% "INDEX"]
eval_df <- eval_df[, !names(eval_df) %in% "TEAM_BATTING_HBP"]
eval_df

Mean/Median Imputation (For Numeric Data)

Instead of removing missing values, fill them with the mean or median:

eval_df$TEAM_BATTING_SO[is.na(eval_df$TEAM_BATTING_SO)] <- mean(eval_df$TEAM_BATTING_SO, na.rm = TRUE)
eval_df$TEAM_BASERUN_SB[is.na(eval_df$TEAM_BASERUN_SB)] <- mean(eval_df$TEAM_BASERUN_SB, na.rm = TRUE)
eval_df$TEAM_BASERUN_CS[is.na(eval_df$TEAM_BASERUN_CS)] <- mean(eval_df$TEAM_BASERUN_CS, na.rm = TRUE)
eval_df$TEAM_FIELDING_DP[is.na(eval_df$TEAM_FIELDING_DP)] <- mean(eval_df$TEAM_FIELDING_DP, na.rm = TRUE)
eval_df$TEAM_PITCHING_SO[is.na(eval_df$TEAM_PITCHING_SO)] <- mean(eval_df$TEAM_PITCHING_SO, na.rm = TRUE)

4. Verifying That Missing Values Are Fixed

sum(is.na(eval_df))  # Total missing values

## [1] 0

colSums(is.na(eval_df))  # Missing values per column

##   TEAM_BATTING_H  TEAM_BATTING_2B  TEAM_BATTING_3B  TEAM_BATTING_HR 
##                0                0                0                0 
##  TEAM_BATTING_BB  TEAM_BATTING_SO  TEAM_BASERUN_SB  TEAM_BASERUN_CS 
##                0                0                0                0 
##  TEAM_PITCHING_H TEAM_PITCHING_HR TEAM_PITCHING_BB TEAM_PITCHING_SO 
##                0                0                0                0 
##  TEAM_FIELDING_E TEAM_FIELDING_DP 
##                0                0

skim(eval_df)

Data summary
Name	eval_df
Number of rows	259
Number of columns	14
_______________________
Column type frequency:
numeric	14
________________________
Group variables	None

Variable type: numeric

skim_variable	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
TEAM_BATTING_H	1	1469.39	150.66	819	1387.0	1455.00	1548.0	2170	▁▂▇▁▁
TEAM_BATTING_2B	1	241.32	49.52	44	210.0	239.00	278.5	376	▁▂▇▇▂
TEAM_BATTING_3B	1	55.91	27.14	14	35.0	52.00	72.0	155	▇▇▃▁▁
TEAM_BATTING_HR	1	95.63	56.33	0	44.5	101.00	135.5	242	▆▅▇▃▁
TEAM_BATTING_BB	1	498.96	120.59	15	436.5	509.00	565.5	792	▁▁▅▇▁
TEAM_BATTING_SO	1	709.34	234.48	0	565.0	709.34	904.5	1268	▁▃▇▆▂
TEAM_BASERUN_SB	1	123.70	91.00	0	60.5	96.00	149.0	580	▇▅▁▁▁
TEAM_BASERUN_CS	1	52.32	18.81	0	44.0	52.32	56.0	154	▁▇▂▁▁
TEAM_PITCHING_H	1	1813.46	1662.91	1155	1426.5	1515.00	1681.0	22768	▇▁▁▁▁
TEAM_PITCHING_HR	1	102.15	57.65	0	52.0	104.00	142.5	336	▇▇▆▁▁
TEAM_PITCHING_BB	1	552.42	172.95	136	471.0	526.00	606.5	2008	▆▇▁▁▁
TEAM_PITCHING_SO	1	799.67	611.78	0	622.5	782.00	927.5	9963	▇▁▁▁▁
TEAM_FIELDING_E	1	249.75	230.90	73	131.0	163.00	252.0	1568	▇▁▁▁▁
TEAM_FIELDING_DP	1	146.06	24.28	69	134.5	146.06	160.5	204	▁▂▇▆▁

Make Predictions Using the Improved_Model:

# Generate predictions using the trained Improved Model
eval_df$PREDICTED_WINS <- predict(improved_model, newdata = eval_df)

# View a sample of predictions
head(eval_df$PREDICTED_WINS)

## [1] 69.15118 71.73961 80.06122 82.52091 75.19106 72.10608

Interpretation of Predicted TARGET_WINS Values

The predicted values represent the expected number of wins for teams based on their batting, pitching, and fielding statistics using the Improved Model.

For example:

A team with a predicted 69.15 wins is expected to win around 69 games in a full season. A team with a predicted 82.52 wins is expected to perform better, likely finishing with around 82-83 wins. The variation in predictions suggests that some teams are stronger than others based on key performance metrics (hits, home runs, walks, strikeouts, fielding errors, etc.).

Evaluating the Accuracy of Predictions Against Actual TARGET_WINS

Now that we have predicted TARGET_WINS, we need to compare these predictions with the actual values in the TARGET_WINS column to assess model accuracy.

Step 1: Compute Error Metrics

# Check structure of the dataset
str(eval_df)

## 'data.frame':    259 obs. of  15 variables:
##  $ TEAM_BATTING_H  : int  1209 1221 1395 1539 1445 1431 1430 1385 1259 1397 ...
##  $ TEAM_BATTING_2B : int  170 151 183 309 203 236 219 158 177 212 ...
##  $ TEAM_BATTING_3B : int  33 29 29 29 68 53 55 42 78 42 ...
##  $ TEAM_BATTING_HR : int  83 88 93 159 5 10 37 33 23 58 ...
##  $ TEAM_BATTING_BB : int  447 516 509 486 95 215 568 356 466 452 ...
##  $ TEAM_BATTING_SO : num  1080 929 816 914 416 377 527 609 689 584 ...
##  $ TEAM_BASERUN_SB : num  62 54 59 148 124 ...
##  $ TEAM_BASERUN_CS : num  50 39 47 57 52.3 ...
##  $ TEAM_PITCHING_H : int  1209 1221 1395 1539 3902 2793 1544 1626 1342 1489 ...
##  $ TEAM_PITCHING_HR: int  83 88 93 159 14 20 40 39 25 62 ...
##  $ TEAM_PITCHING_BB: int  447 516 509 486 257 420 613 418 497 482 ...
##  $ TEAM_PITCHING_SO: num  1080 929 816 914 1123 ...
##  $ TEAM_FIELDING_E : int  140 135 156 124 616 572 490 328 226 184 ...
##  $ TEAM_FIELDING_DP: num  156 164 153 154 130 ...
##  $ PREDICTED_WINS  : num  69.2 71.7 80.1 82.5 75.2 ...

# Check if ACTUAL_WINS and PREDICTED_WINS are numeric
is.numeric(eval_df$ACTUAL_WINS)

## [1] FALSE

is.numeric(eval_df$PREDICTED_WINS)

## [1] TRUE

Step 2: Add an INDEX Column to Each Dataset

# Add a row index to both datasets
train_df$INDEX <- seq_len(nrow(train_df))  
eval_df$INDEX <- seq_len(nrow(eval_df))

Step 3: Align and Merge the Datasets

# First, check if both datasets have a common identifier (like INDEX)
head(train_df$INDEX)

## [1] 1 2 3 4 5 6

head(eval_df$INDEX)

## [1] 1 2 3 4 5 6

# Merge train_df and eval_df using INDEX (if applicable)
merged_df <- merge(train_df[, c("INDEX", "TARGET_WINS")], 
                   eval_df[, c("INDEX", "PREDICTED_WINS")], 
                   by = "INDEX", all = FALSE)

# Now check if both columns have equal length
nrow(merged_df)

## [1] 259

Ensure data consistency before calculating accuracy metrics. Convert to numeric to avoid errors in correlation or calculations.

Step 4: Compute Accuracy Metrics

# Load necessary libraries
library(Metrics)  # For MAE and RMSE

## 
## Attaching package: 'Metrics'

## The following objects are masked from 'package:caret':
## 
##     precision, recall

## The following object is masked from 'package:forecast':
## 
##     accuracy

library(ggplot2)  # For visualizations

# Compute MAE, RMSE, and R²
mae <- mae(merged_df$TARGET_WINS, merged_df$PREDICTED_WINS)  
rmse <- rmse(merged_df$TARGET_WINS, merged_df$PREDICTED_WINS)  
r_squared <- cor(merged_df$TARGET_WINS, merged_df$PREDICTED_WINS)^2  

# Print results
cat("Model Accuracy Metrics:\n")

## Model Accuracy Metrics:

cat("Mean Absolute Error (MAE):", round(mae, 2), "\n")

## Mean Absolute Error (MAE): 13.87

cat("Root Mean Squared Error (RMSE):", round(rmse, 2), "\n")

## Root Mean Squared Error (RMSE): 17.38

cat("R-Squared (R²):", round(r_squared, 4), "\n")

## R-Squared (R²): 0.001

Interpretation of Model Accuracy Metrics

The evaluation results indicate that our Improved Model is not performing well in predicting TARGET_WINS.

The Mean Absolute Error (MAE) of 13.87 suggests that, on average, predictions deviate by about 14 wins per team, which is quite high.

Additionally, the Root Mean Squared Error (RMSE) of 17.38 implies that some predictions have even larger errors, highlighting potential inconsistencies or missing key predictors.

Most concerning is the R-Squared (R²) value of 0.001, which means that the model explains almost none of the variation in team wins—essentially, the predictions are no better than random guesses.

This suggests that the model may be overfitting the training data and failing to generalize, or that it lacks critical predictive features.

To improve performance, we should consider adding more relevant predictors (such as ERA or OBP), removing unimportant or noisy variables, and potentially using alternative mo

Assignment1_Data621

Mubashira Qari

2025-02-09

Load Libraries

Data Exploration

Loading Datasets

Understanding the TARGET_WINS Field in the Moneyball Dataset:

Viewing Column Names

Viewing Field Values:

Creating Dataframe

Summary of Datasets

Calculate mean, median, and standard deviation for key numerical columns

Analysis of the training Dataset Before Fitting a Multiple Linear Regression Model:

1. Understanding the Target Variable (TARGET_WINS)

2. Missing Values

Actionable Steps:

3. Potential Outliers & Data Issues

Actionable Steps:

4. Feature Selection Considerations:

Multicollinearity Check Needed:

5. Data Cleaning Recommendations Before Fitting the Model

Conclusion:

Visualizing Training Dataset:

Analysis of the Box Plot for TARGET_WINS (Team Wins Distribution)

1. Median and Spread of Wins

2. Box Plot – for Outlier Detection & Distribution Analysis:

Consideration for Regression?

3. Data Skewness and Symmetry

1. Remove Columns with Too Many Missing Values

2. Remove Rows with Missing Target (TARGET_WINS)

3. Create a “Missing Indicator” Column

4. Verifying That Missing Values Are Fixed

Handling Additional Missing Values

Reasons for Removing Missing Values

Preventing Bias in Model Predictions

Alternative Solutions Instead of na.omit()

Mean/Median Imputation (For Numeric Data)

Building Three Multiple Linear Regression Models with Manual Variable Selection

Model 1: Base Baseball Stats Model

Why were these variables selected?

Interpreting the Coefficients of the Regression Model

Key Metric Interpretation:

Conclusion:

Implementing Improvement in the Model:

Step 1: Check for Multicollinearity (VIF Test)

Run VIF test in R:

Interpretation

Step 2: Test Interaction Term (TEAM_BATTING_HR * TEAM_BATTING_BB)

Interpretation:

Visualizing the Interaction Effect:

Adding TEAM_BATTING_2B (Doubles) to the Model:

Interpretation:

Improvement After Adding TEAM_BATTING_2B (Doubles)?

Key Findings

Model 2: High-Impact Features Model (Based on Correlation & VIF)

Why these variables?

Model 3: Log-Transformed Model (Handling Skewness & Outliers)

Why log transformation?

Comparision between three Models:

Model 4: Variable Selection Using Backward Selection

Model 5: Best Subset Selection

Model 6: Cross Validation

Model 6: Ridge Regression

Conclusion:

Select Model:

Final Analysis

Assumptions of Multiple Linear Regression:

Data Preparation:

Removing Mostly NA and INDEX Column:

Mean/Median Imputation (For Numeric Data)

4. Verifying That Missing Values Are Fixed

Make Predictions Using the Improved_Model:

Interpretation of Predicted TARGET_WINS Values

Evaluating the Accuracy of Predictions Against Actual TARGET_WINS

Step 1: Compute Error Metrics

Step 2: Add an INDEX Column to Each Dataset

Step 3: Align and Merge the Datasets

Step 4: Compute Accuracy Metrics

Interpretation of Model Accuracy Metrics